Subspace Clustering Reloaded: Sparse vs. Dense Representations

State-of-the-art methods for learning unions of subspaces from a collection of data leverage sparsity to form representations of each vector in the dataset with respect to the remaining vectors in the dataset. The resulting sparse representations can be used to form a subspace affinity matrix to cluster the data into their respective subspaces. While sparsity-driven methods for subspace clustering provide advantages over traditional nearest neighbor-based approaches, sparse representations often produce affinity matrices with weakly connected components that are difficult to cluster, even with state-of-the-art clustering methods. In this work, we propose a new algorithm that employs dense (least-squares) representations to extract the subspace affinity between vectors in the dataset. We demonstrate the advantages of the proposed dense subspace clustering algorithm over state-of-the-art sparsity-driven methods on real and synthetic data. I. SPARSE SUBSPACE CLUSTERING Unions of linear subspaces are a widely used signal model for representing collections of high-dimensional data, such as images of faces acquired under varying illumination conditions, motion trajectories from different objects, or local field potentials in brainmachine-interface applications [1], [2]. In order to use this signal model, the subspaces that the collection of data live upon must be learned from the data by performing subspace clustering—learning subspaces present in the data and clustering the data based upon the subspace membership of each vector. Subspace clustering is challenging due to the fact that subspace estimation and segmentation must be performed simultaneously. Sparse subspace clustering (SSC) has been shown to yield stateof-the-art performance on both synthetic and numerous real-world image datasets [1]. In SSC, a sparse representation of a signal in a collection of data is formed with respect to the remaining signals in the same dataset. The idea underlying this approach is that signals from the same subspace cluster will use one another in their sparse representations, thus revealing which points belong to the same subspace. SSC provides a powerful alternative to nearest neighbor (NN)-based approaches to subspace clustering and enables the derivation of guarantees that describe when convex methods [1], [3] or greedy algorithms [2] yield representations that only contain points from the same subspace. Formally, SSC computes sparse representations ci ∈ R for each point yi ∈ R, i = 1, . . . , d, via the following optimization problem: ci = argminc̃∈Rd ‖c̃‖1 subject to yi = ∑ j : j 6=i yj c̃j , (1) which are used to construct an affinity matrix C = [ c1· · · cd ]. Spectral clustering is then performed on the graph Laplacian of W = |C|+ |C | to segment the data into subspace clusters [2]. II. DENSE SUBSPACE CLUSTERING While sparse representations result in affinity matrices that contain a small number of edges in the graph linking signals from different subspaces, recovering subspace clusters from the affinity matrices obtained via SSC is challenging due to the fact that sparse representations often produce weakly connected components between signals in the same subspace. To circumvent this issue, we propose a novel method for subspace clustering that is based on forming dense leastsquares representations from the data. This method, which we refer 1We replace the `1 norm in (1) with the `2 norm resulting in (typically dense) least-squares representations of the data. DSC NN SSC